Recently, the performance and functionality of digital cameras and digital movie cameras that use some solid-state image sensor such as a CCD and a CMOS (which will be sometimes simply referred to herein as an “image sensor”) have been enhanced to an astonishing degree. In particular, the size of a pixel structure for use in an image sensor has been further reduced these days thanks to rapid development of semiconductor device processing technologies, thus getting an even greater number of pixels and drivers integrated together in an image sensor. As a result, the resolution of an image sensor has lately increased rapidly from around one million pixels to ten million or more pixels in a matter of few years. On top of that, the quality of an image captured has also been improved significantly as well. As for display devices, on the other hand, LCD and plasma displays with a reduced depth now provide high-resolution and high-contrast images, thus realizing high performance without taking up too much space. And such video quality improvement trends are now spreading from 2D images to 3D images. Furthermore, image capture devices that can estimate the depth of a subject have just started to be developed.
As far as getting subject's depth information is concerned, there is a technique for getting depth information using a single-lens camera with multiple micro lenses and for changing the focus position of the image captured freely based on that information. Such a technique is called “light field photography” and a single-lens camera that uses such a technique is called a “light field camera”. In a light field camera, a number of micro lenses are arranged on an image sensor. Each of those micro lenses is arranged so as to cover a plurality of pixels. By calculating information about the direction of incoming light based on the image information gotten through the image capturing session, the subject's depth can be estimated. Such a camera is disclosed in Non-Patent Document No. 1, for example.
The light field camera can calculate depth information. But its resolution is determined by the number of micro lenses and should be lower than the resolution determined by the number of pixels of the image sensor, which is a problem. Thus, to overcome such a problem, Patent Document No. 1 discloses a technique for increasing the resolution using two image capturing systems. According to such a technique, the incoming light is split into two divided incoming light beams, which are imaged by two image capturing systems, of which the groups of micro lenses are arranged so as to spatially shift from each other by a half pitch, and then the images captured in this manner are synthesized together, thereby increasing the resolution.
On the other hand, Patent Document No. 2 discloses a technique for changing the modes of operation from a normal shooting mode into the light field photography mode, or vice versa, using a single image capturing system. According to this technique, a micro lens, of which the focal length varies according to the voltage applied, is used. Specifically, the focal length of the micro lens is set to be the infinity in the former mode and set to be a predetermined distance in the latter mode. By adopting such a mechanism, an image with high resolution and depth information can be obtained.